Software is generally the expression of computer readable code with which a user interacts. A software product may be a collection of this code that is released to consumers. Before being released to consumers as a software product, software undergoes many stages of development including requirements gathering, design, coding, testing, and deployment.
Requirements gathering may encompass tasks used to determine the needs or conditions for a new or modified product. Design may include tasks implementing solutions to problems which may be determined during the requirements stage. Coding may include tasks for generating the machine readable code necessary to cause a computer processor to perform the solutions implemented in the design stage. Testing may include tasks for investigating the quality of the code generated. Deployment may include tasks for making software available as a software product.
The machine code of a software product may be further understood as instructions that direct a computer to perform a task. If the code correctly expresses the software, then the consumers may use their software product with little disruption. The code, however, may contain a fault such as an incorrect step, process, or definition in the code. The fault may then propagate to a functional field failure such as a script error appearing on a display or the abortion of a software application.
Production faults occurring in deployed software applications are one of the worst negative factors that destroys software products. A major problem is that production faults are not known by definition when the software product application is deployed. A functional field failure generally occurs after a software product is released to consumers. Thus, only when field failures occur in a deployed application can a person (i.e., programmer) begin to analyze the symptoms of failures to determine what faults can cause them. Examples of symptoms of failures include but not limited to incorrect output values, program crashes and computations that take much more time than they are supposed to, possibly indicating infinite loops.
The cost for detecting and repairing a fault for functional field failure after release to consumers is estimated to be approximately 300 times higher than during the phases of software development. In attempt to reduce costs, many have developed manual and automatic methods for detecting and repairing these faults.
According to one method, intuitive testing is used to detect and repair faults. Using this method, the person may develop an intuition about how different input values and changes to code affect the software's behavior. It may take a person long periods of time to develop an intuition. The person may then surmise probable faults which result in functional field failures. Surmising probable faults may only be accomplished after repeated trial and error. Methods like this may be difficult and time consuming.
Other methods attempt to detect and repair faults automatically. Some of these automatic methods may further use fault localization. Fault localization may include a collection of statistical techniques and algorithms for testing the behavior of software. When tested in this way, the behavior of software may be correlated with functional field failures by ranking statements of code according to the likelihood the statements contain faults. Some methods including fault localization may also be resource intensive and time consuming.
In additional methods, the values of what are termed “program predicates” may be collected from running the software. A “predicate” is generally a choice to execute or not execute an instruction. In one example, a Boolean predicate may give a value of true or false for a conditional expression. A statistical model may then be built by contrasting the values of program predicates for successful runs with those of unsuccessful runs. As a result, methods such as these may require large amounts of state information and data from both failed runs and successful runs.
Added methods may also use what are termed “oracles” to test software. An “oracle” is generally a mechanism for testing whether the software behaves correctly for specific outputs. Methods using an oracle may compare the outputs of software under testing (SUT) to outputs predicted by the oracle. Oracles often operate separately from the SUT. Creating oracles automatically may be one of the most challenging problems of software testing.
Therefore, a need exists for an efficient system and methods for detecting and repairing faults for functional field failure by automatically localizing faults without collecting extensive runtime data, without requiring manually developed oracles, without deploying instrumented applications, and without collecting large amounts of state information. This invention satisfies that need.